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Do Social Science Students Value Empirical Research? Answers from a 
Canadian and Dutch Investigation 


Abstract 

Although students in the social sciences perceive quantitative methods courses negatively, this need not mean 
that they devalue empirical research, or lack capacity to become informed consumers of research. To explore 
this possibility, we administered two measures to Canadian students (n = 194) enrolled in first-year social 
science courses, and to Dutch criminology students (n =156) enrolled in a bachelor’s or masters program. 
While students in each country expressed low interest in engaging in research, they expressed significantly 
higher appreciation of the value of research. Further, we found a small-medium positive correlation between 
education and appreciation of research in the Dutch sample. We propose that while experiential research 
activities have little impact on students’ interest in conducting research, they likely add to students’ 
appreciation of the importance of research. 
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Abstract 

Although students in the social sciences perceive quantitative methods courses negatively, 
this need not mean that they devalue empirical research, or lack capacity to become 
informed consumers of research. To explore this possibility, we administered two measures 
to Canadian students (n = 194) enrolled in first-year social science courses, and to Dutch 
criminology students (n =156) enrolled in a bachelor's or master's program. While students 
in each country expressed low interest in engaging in research, they expressed significantly 
higher appreciation of the value of research. Further, we found a small-medium positive 
correlation between education and appreciation of research in the Dutch sample. We 
propose that while experiential research activities have little impact on students' interest in 
conducting research, they likely add to students' appreciation of the importance of research. 

Keywords: students' perceptions of research; assessing interest in research; assessing 
appreciation of research; influencing perceptions of research 


Introduction 

We suspect that few social science instructors, particularly those who teach quantitative 
methods, are surprised by the literature which shows that undergraduate students lack 
interest in our attempts to develop their quantitative literacy and capacity to become 
informed consumers of research. Criminal justice students, for instance, expressed 
disinterest in taking research methods courses (Briggs, Brown, Gardner & Davidson, 2009), 
social work students expressed anxiety over taking methods courses (Secret, Ford & Rompf, 
2003), while psychology students considered "human interest" courses more appealing than 
methods courses (Rajecki, Appleby, Williams, Johnson & Jeschke, 2004), and delayed 
enrolling in statistics courses (Lauer, Rajecki & Minke, 2006). This lack of interest may also 
contribute to the difficulties experienced by students in quantitative methods courses: 
Education, psychology, social policy and sociology students enrolled in research methods 
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courses in Finland and the US, for example, described the courses as "difficult" (Murtonen, 
2005; Murtonen & Lehtinen, 2003); and criminal justice students performed more poorly 
than biology and nursing students in statistics courses (Proctor, 2006). 

At least two variables appear relevant in understanding students' perceptions of quantitative 
courses. First, personality may contribute to student disinterest in the courses. Openness 
to experience is a well-researched personality trait that captures individual differences in 
curiosity, and preference for intellectual pursuits (Costa & McCrae, 1992). The trait is 
normally distributed, with one in six showing high openness to experience. Importantly, the 
trait is associated with interest in research, at least in psychology majors. Vittengl et al. 
(2004) found a moderate positive correlation between interest in research activities and 
openness to experience, and Holmes and Beins (2009) found a high positive correlation 
between interest in research activities and need for cognition (Cacioppo, Petty & Kao, 

1984), which overlaps theoretically and empirically with openness to experience (Sadowski 
& Cogburn, 1997). Thus, perhaps only a minority of students have high intrinsic motivation 
to learn about, and engage in, research. A second explanation of disinterest in research is 
found in the proposition that students in the social sciences commonly consider capacity for 
research irrelevant to their career goals, presumably viewing research capacity as relevant 
only to those seeking careers as "scientists" or academic researchers (Manning, Zachar, Ray 
& LoBello, 2006; Maxfield & Babbie, 2008). In support of this proposition, Briggs et al. 
(2009) found a high positive correlation between disinterest in research and perceived 
relevance of research in criminal justice students, while Sizemore and Lewandowski (2009) 
reported a similarly high positive correlation between attitudes towards research and 
perceived utility of research in psychology majors. 

Should we be concerned about students' disinterest in quantitative courses? Given that 
fostering students' capacity for quantitative reasoning is a common criterion in 
baccalaureate curriculum guidelines and accreditation standards (e.g., Academy of Criminal 
Justice Sciences, 2005; American Psychological Association, 2007) we believe there are 
reasonable grounds for concern - clearly, we expect students to grasp the importance of 
empirical methods. However, while disinterest in quantitative courses is certainly 
disappointing, it need not follow that this disinterest is indicative of a general devaluing of 
research, or failure to appreciate the significance and utility of research findings. We would 
suggest that although the majority of our students may have limited intrinsic interest in 
learning how to conduct research or use statistics, and may not expect to conduct research 
in their careers, it is quite conceivable that they may, nevertheless, consider the research- 
based literature an important source of information about social science issues. Indeed, 
given that encouraging students to value empirical evidence over anecdote and intuition is 
one of the most important overarching goals in social science education, it would be 
disturbing if students did not, in fact, develop an appreciation of research findings during 
the course of their studies. 

There is some empirical support for distinguishing between students' interest in conducting 
research and the extent to which they recognize the importance of research. Vittengl et al. 
(2004), for example, asked psychology majors to read the titles and abstracts from ten 
published articles, and indicate their interest in engaging in five follow-up activities ordered 
on a passive-active continuum. Although students expressed low interest in active or high 
commitment activities such as conducting research on the abstract topic, they expressed 
moderate interest in passive or low commitment activities such as reading the rest of the 
article. More recently, McConnell and Marton (2011) examined the influence of hands-on 
research experience on students' perceptions of research. While first-year psychology 
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students showed no increase in their interest in engaging in further research activities as a 
result of participating in a community research project, they provided qualitative feedback 
suggesting that the project enhanced their appreciation of the nature and value of research 

Encouraged by these two studies, we decided to more formally explore the distinction 
between interest in the mechanics of research, and appreciation of the importance of 
research. In doing so, we identified three related research questions. First, given that 
students often express disinterest in research, and given our assumption that appreciation 
of research is responsive to an instructional emphasis on the importance of empirical 
methods, we wanted to know if students generally express higher appreciation of research 
than interest in research. Second, given that introductory social science courses represent 
an important opportunity to socialize students to the significance of empirical research, 
reflected in the fact that most introductory textbooks cover research methods (e.g., Griggs 
& Marek, 2001; Rhineberger, 2006), we wanted to know if differences between interest in 
research and appreciation of research can be detected in first-year students. Third, we 
wanted to know if interest in research and appreciation of research are influenced 
differentially by educational experience. Given the documented association between 
interest in research and personality (Holmes & Beins, 2009; Vittengl et al., 2004), we 
assumed that interest in research is relatively stable, and that we would find no association 
between education and interest. However, given our assumption that students are 
reasonably receptive to our efforts to cultivate appreciation of empirical methods, we 
concluded that appreciation of research should increase over time in response to our 
ongoing efforts, and thus we predicted that we would find a positive association between 
education and appreciation of research. 

We examined our research questions in a two-stage study. In the first, we administered 
McConnell and Marton's (2011) interest in research scale and a new scale measuring 
appreciation of research to a large sample of students registered in first-year social science 
courses at a Canadian community college. In the second, we administered both scales to 
criminology students registered in a three-year bachelor's of arts program and a one year 
master's of arts program at a large university in the Netherlands. 


Method 


Participants 

Canada 

One hundred and ninety-four students in nine social science courses participated in the 
study: 21 in Introduction to Criminology; 13 in Introduction to the Criminal Justice System; 
19 in Introductory Psychology I; 83 in three sections of Introductory Psychology II; and 58 
in three sections of Introduction to Sociology II. These students represented almost all 
students registered in first-year social science courses at the college's principal campus. 
Their mean age was 25, and 76% were female. 

Students who take first-year social science courses at the college are not asked to declare a 
major. They can only take research methods courses in their second year. The majority 
transfer after their first or second year to one of British Columbia's universities, where they 
perform as well academically as those who start out at the universities (e.g., British 
Columbia Council on Admissions and Transfer, 2009). 
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Netherlands 

One hundred and fifty-six students, representing approximately 50% of all students in both 
the bachelor's and master's programs, participated in the study: 43 first-year students; 42 
second-year students; 44 third-year students; and 27 master's students. The majority of 
students in the master's program graduated from the university's bachelor's program. The 
mean age in each of the four years was 20, 21, 23 and 23, and the proportion of females in 
each year was 81%, 74%, 77% and 74%. 

The bachelor's and master's programs emphasize exposure to research methods. First-year 
bachelor's students complete a Methods and Techniques course which includes hands-on 
exposure to basic techniques (e.g., interviewing, content analysis), and conduct a small- 
group project in a Descriptive Criminology laboratory course; second-year students 
complete a Methods and Statistics course, and gain hands-on exposure to methodological 
techniques in an Explanatory Criminology laboratory course and in Introduction to 
Psychology and Law; and third-year students complete a bachelor's project course, prepare 
an empirically-based poster for a Nature, Extent and Consequences of Crime course, and 
submit a bachelor's empirical thesis. Master's students submit an empirical thesis. 

Procedure and Measures 

Students in both countries completed a brief demographic questionnaire and the two 
research scales - the six-item "Interest in Engaging in Research Scale" (IERS) and the six- 
item "Appreciation of Research Scale" (ARS) - in single in-class administrations towards the 
end of the academic year. Participation was voluntary, and students had the right to 
withdraw at any time without penalty. 

Interest in Engaging in Research Scale (IERS) 

The IERS measures interest in engaging in research opportunities available to social science 
undergraduates (e.g., "I would like to work with other students on a research project", "I 
would be interested in applying for a job as a research assistant"), and includes an item on 
interest in taking methods courses. The six items are positively-keyed, and the scale uses a 
five-point response format ranging from 1 ( strongly disagree ) to 5 ( strongly agree). Thus, 
higher scores indicate higher interest. McConnell and Marton (2011) reported Cronbach's 
alpha coefficients (which provide an indication of the extent to which all six scale items 
measure the same underlying construct) of .83 and .89 for their pre and posttest 
administrations of the scale. The alpha coefficients for the current study were .88 for the 
Canadian sample, and .77 for the combined Dutch sample. Given that a coefficient of .7 
and above is generally considered an acceptable indicator of reliability in social science 
research, these coefficients indicate high internal reliability. 

Appreciation of Research Scale (ARS) 

We developed the ARS to measure appreciation of the value of empirical research. Three 
items measure value placed on empirical methods (e.g., "Research is necessary to 
understand human behaviour"), while three measure interest in research outcomes (e.g., "I 
have discussed social science research with friends and relatives"). Five items are 
positively-keyed, one item is negatively-keyed, and the scale uses a five-point response 
format ranging from 1 ( strongly disagree) to 5 ( strongly agree). Again, higher scores 
indicate higher appreciation. The alpha coefficients were .74 for the Canadian sample, and 
.71 for the Dutch. These coefficients indicate acceptable internal reliability. 
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Results 

The IERS and ARS scales were significantly correlated in both the Canadian and Dutch 
samples, r(194) = .54, p < .001; and r(156) = .46, p < .001, and ARS scores were 
significantly higher than IERS scores in each sample, t(192) = 9.47, p < .001, d = .68; and 
t(155) = 13.57, p < .001, d - 1.13. Generally, for every Canadian and Dutch student who 
expressed interest in engaging in research, another did not. Moreover, the majority of 
those who expressed interest did so only mildly or moderately, with only 12% of the 
Canadian sample and 3% of the Dutch sample expressing high interest in research (i.e., 
obtaining a mean scale score higher than four). However, over 85% in each sample 
expressed at least some appreciation of the value of research (i.e., obtaining a mean scale 
score higher than three), and 27% of the Canadian sample and 18% of the Dutch sample 
expressed high appreciation. 

Table 1 shows the descriptive statistics for the Canadian sample and the breakdown for each 
curriculum year of the Dutch sample. IERS and ARS scores for the Canadian sample and 
the first-year Dutch bachelor's students were comparable, albeit with slightly higher interest 
in research and appreciation of research in the Canadian sample. IERS scores were almost 
equivalent in each year in the Dutch sample. ARS scores were also almost 
equivalent in the first and second years of the Dutch bachelor's program, and increased in 
the third year of the program, and into the master's program. 


Table 1 . Results for the IERS and ARS by country and curriculum year 



IERS 

ARS 



Cohen's 


M (SD) 

M (SD) 

df 

t 

d 

Canadian students 

3.24 (.82) 

3.72 (.57) 

192 

9.47*** 

.68 

Dutch first-year students 

3.00 (.58) 

3.54 (.44) 

42 

5 96*** 

1.05 

Dutch second-year students 

2.94 (.76) 

3.51 (.50) 

41 

6.27*** 

.89 

Dutch third-year students 

2.96 (.69) 

3.73 (.60) 

43 

8.28*** 

1.19 

Dutch master's students 

2.98 (.71) 

3.99 (.44) 

26 

7.37*** 

1.71 


*** p <.001 


We examined the relationship between education and IERS and ARS scores by computing 
zero-order correlations between education and each scale in the Dutch sample. To quantify 
education on a continuous scale, we assigned undergraduates one, two or three years of 
education depending on their year in the program, and master's students four years of 
education. The correlation between education and IERS scores was non-significant, r(156) 
= .01, p = .898. However, there was a significant small-medium correlation between 
education and ARS scores, r(156) = .30, p < .001. 
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Discussion 

The results provide answers to our three research questions. First, we wanted to know if 
students who express low interest in engaging in research nevertheless recognize the 
importance of research and empirical evidence. The significant and substantial differences 
between interest in engaging in research and appreciation of research found in both the 
Canadian and Dutch samples show that this was indeed the case: While the average student 
in both samples was relatively uninterested in engaging in research, she expressed some 
appreciation of research, placing value on empirical methods, and expressing interest in 
learning about research outcomes. These findings are consistent with the disinterest in 
quantitative courses generally observed in social science students (Briggs et al., 2009; 

Lauer et al., 2006; Rajecki et al., 2004; Secret et al., 2003), and with our proposal that one 
can meaningfully distinguish between students' interest in conducting research and their 
appreciation of the value of empirical methods. Second, given the emphasis placed in 
introductory social science courses on the importance of empirical methods, we wanted to 
know if a difference between interest in engaging in research and appreciation of research 
could be detected in first-year students. Again, we found that while both the Canadian 
students and the Dutch first-year students expressed low interest in engaging in research, 
they expressed significantly higher appreciation of research. Third, we wanted to know if 
interest in engaging in research and appreciation of research are associated with education. 
The non-significant correlation between education and interest in engaging in research 
found in the Dutch sample is consistent with the hypothesis that, influenced by personality, 
interest in engaging in research is relatively stable (Holmes & Beins, 2009; Vittengl et al., 
2004), while the small-medium positive correlation found for education and appreciation of 
research is consistent with our assumption that appreciation of research increases in 
response to an ongoing instructional emphasis on the importance of empirical methods. 

While our study is methodologically strong in its use of reliable outcome measures and two 
large cross-national student samples, there are, however, some methodological 
weaknesses. First, although the term "research" is undefined in the two scales employed in 
our study, we assumed that students would understand that we were referring to 
quantitative research. Murtonen (2005) has observed that some social science 
undergraduates express a preference for qualitative over quantitative research. If students 
in our sample did in fact assume that we were referring to quantitative research but held a 
preference for qualitative research, then we may have underestimated their interest in 
research. 

Second, and more importantly, while we are inclined to interpret our results as showing the 
influence of education on the development of appreciation of research in the Dutch sample, 
the correlational nature of our design does not, of course, allow us to draw a causal 
conclusion. Proving that we can in fact influence students' appreciation of research, and 
showing that our obtained correlation between education and appreciation of research was 
not due to a third, unmeasured variable such as maturation, will require a more rigorous 
focus on factors that seem likely to influence their appreciation. Our data suggest that the 
experiential research activities provided to the Dutch students in each curriculum year had 
no discernible impact on their interest in conducting further research. However, we suspect 
that research activities can contribute to the development of students' appreciation of 
research. This proposition finds support in McConnell and Marton's (2011) qualitative 
analysis of first-year students' impressions of the benefits of participating in a research 
project, and in studies that have analysed students' impressions of the value of including 
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experiential activities in research methods courses. Students who participated in a 
community-based research project, for example, reported increased understanding of 
research (Chapdelaine & Chapman, 1999), while those who conducted self-directed projects 
reported increased interest in research (Ball & Pelco, 2006). 

We therefore encourage investigations using psychometric measures to clarify the effect of 
participating in research projects on students' appreciation of research. One might, for 
example, measure the influence of methods courses that include a hands-on project, or the 
influence of completing a bachelor's thesis. Further, given that volunteering as a participant 
in a research project offers insight into the nature of research (Payne & Chappell, 2008) and 
is viewed favourably by students (Bowman & Waite, 2003), it may also be productive to 
examine the impact of such participation on students' appreciation of research. Including 
control or comparison groups in these investigations (e.g., comparing sections of methods 
courses that include experiential activities with those that do not) will clarify the extent to 
which research experience contributes to students' appreciation of research over and above 
the general emphasis on the importance of empirical methods provided in our courses. 


Conclusion 

We are not perturbed by the fact that the majority of the 350 students who participated in 
this study were relatively uninterested in conducting research. In reality, only a small 
minority of social science students pursue academic careers, or elect to work in settings 
where they will be expected to conduct research. Presumably, these students are drawn 
from the small pool of those with high intrinsic motivation to conduct research. Many, 
instead, choose careers where basic quantitative literacy and the capacity to consume 
rather than produce research are desired assets. Given that many of the students in this 
study possess or will acquire at least rudimentary quantitative skills as a consequence of 
taking methods and statistics courses, and given that most of them place some value on 
empirical evidence, we believe there is good reason to assume that a significant proportion 
are well prepared to function successfully in their chosen careers. 
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